Methods and systems for transition-coded media, measuring engagement of transition-coded media, and distribution of components of transition-coded media

ABSTRACT

Disclosed are methods, systems, and non-transitory computer-readable medium for transition-coded media, measuring engagement of transition-coded media, and distribution of components of transition-coded media. For instance, the method may include: obtaining a media file; determining whether the media file includes transition-coded media; in response to determining the media file includes the transition-coded media, setting up a dynamic media environment; detecting whether a user is interacting with the transition-coded media; and in response to detecting the user is interacting with the transition-coded media, executing the transition-coded media with the dynamic media environment.

TECHNICAL FIELD

Various embodiments of the present disclosure relate generally tomethods and systems for media and, more particularly, to methods andsystems for transition-coded media, measuring engagement oftransition-coded media, and distribution of components oftransition-coded media.

BACKGROUND

Generally, a user may interact with media, such as listen to media,read/view media, feel media, etc. Further, content consumption of media(e.g., entertainment, advertising, information, etc.) may be movingtowards 3D and 3D+ consumption experiences. For instance, print media(e.g., books/advertising material) may increasingly be available in adigital format, while numerous other media types (e.g., audio, visual,and audio/visual content) may be readily available in digital format.One challenge in content consumption in 3D and 3D+ consumptionexperiences may be understanding and/or enhancing content consumption.Moreover, another challenge in content consumption in 3D and 3D+consumption experiences may be determining a user's reaction to thecontent. Furthermore, another challenge in content consumption in 3D and3D+ consumption experiences may be in developing the media itself sothat a desired user reaction is achieved.

The present disclosure is directed to overcoming one or more of theseabove-referenced challenges. The background description provided hereinis for the purpose of generally presenting the context of thedisclosure. Unless otherwise indicated herein, the materials describedin this section are not prior art to the claims in this application andare not admitted to be prior art, or suggestions of the prior art, byinclusion in this section.

SUMMARY

According to certain aspects of the disclosure, systems and methods aredisclosed for transition-coded media. For instance, a method mayinclude: obtaining a media file; determining whether the media fileincludes transition-coded media; in response to determining the mediafile includes the transition-coded media, setting up a dynamic mediaenvironment; detecting whether a user is interacting with thetransition-coded media; and in response to detecting the user isinteracting with the transition-coded media, executing thetransition-coded media with the dynamic media environment.

A system may include a memory storing instructions, and a processorexecuting the instructions to perform a process. The process mayinclude: obtaining a media file; determining whether the media fileincludes transition-coded media; in response to determining the mediafile includes the transition-coded media, setting up a dynamic mediaenvironment by: obtaining location data for the dynamic mediaenvironment, the location data including information about one or moredynamic media environment enabled devices, and initializing the one ormore dynamic media environment enabled devices; detecting whether a useris interacting with the transition-coded media; and in response todetecting the user is interacting with the transition-coded media,executing the transition-coded media with the dynamic media environment.

A method may include: obtaining a media file; determining whether themedia file includes transition-coded media; in response to determiningthe media file includes the transition-coded media, setting up a dynamicmedia environment; detecting whether a user is interacting with thetransition-coded media; and in response to detecting the user isinteracting with the transition-coded media, executing thetransition-coded media with the dynamic media environment by: tracking aprogress of a media element of the transition-coded media and a userstate; based on the tracking of the progress of the media element andthe user state, determining whether a trigger event occurs; in responseto determining the trigger event occurs, determining an transitionaction based on the trigger event; and performing the transition action.

According to certain aspects of the disclosure, systems and methods aredisclosed for measuring engagement of transition-coded media. Forinstance, a method may include: obtaining a media file; determiningwhether the media file includes transition-coded media; in response todetermining the media file includes the transition-coded media,establishing a dynamic media environment; detecting whether a user isinteracting with the transition-coded media; in response to detectingthe user is interacting with the transition-coded media, executing thetransition-coded media with the dynamic media environment; and whileexecuting the transition-coded media, performing a baseline engagementprocess to obtain a baseline engagement of the user.

A system may include a memory storing instructions, and a processorexecuting the instructions to perform a process. The process mayinclude: obtaining a media file; determining whether the media fileincludes transition-coded media; in response to determining the mediafile includes the transition-coded media, establishing a dynamic mediaenvironment; detecting whether a user is interacting with thetransition-coded media; in response to detecting the user is interactingwith the transition-coded media, executing the transition-coded mediawith the dynamic media environment; and while executing thetransition-coded media: obtaining a baseline engagement of the user; andperforming a reaction trigger process.

A method may include: obtaining a media file; determining whether themedia file includes transition-coded media; in response to determiningthe media file includes the transition-coded media, establishing adynamic media environment; detecting whether a user is interacting withthe transition-coded media; in response to detecting the user isinteracting with the transition-coded media, executing thetransition-coded media with the dynamic media environment; and whileexecuting the transition-coded media: performing a baseline engagementprocess to obtain a baseline engagement; and after obtaining thebaseline engagement, performing a reaction trigger process.

According to certain aspects of the disclosure, systems and methods aredisclosed for distribution of components of transition-coded media. Forinstance, a method may include: hosting a service for components of oneor more transition-coded media, the components including one or more of:engagement index information for a type of media, engagement profilesfor effects and/or dynamic processes, registration information, ormodification information; receiving a request from a user device;determining whether the request includes a search request, a componentrequest, or a data request; and based on the determining, performing oneor more of a search process, a component serve process, and/or a dataserve process, in accordance with the request including the searchrequest, the component request, and/or the data request, respectively.

A system may include a memory storing instructions, and a processorexecuting the instructions to perform a process. The process mayinclude: hosting a service for components of one or moretransition-coded media, the components including one or more of:engagement index information for a type of media, engagement profilesfor effects and/or dynamic processes, registration information, ormodification information; receiving a data input message from a userdevice; extracting a component identifier and new media engagementinformation from the data input message; determining whether one or moreof the components has a matching identifier to the component identifier;in response to determining the one or more of the components has amatching identifier, obtaining media engagement data for the one or moreof the components that have the matching identifier; and in response toobtaining the media engagement data, updating the media engagement databased on the new media engagement information.

A method may include: hosting the service for the components of one ormore transition-coded media; receiving a data input message from a userdevice; in response to receiving the data input message, performing amedia engagement update process to update media engagement informationfor one or more components of the components; receiving a request fromanother user device; in response to receiving the request, performing areporting process to report the updated media engagement information forthe one or more components of the components.

Additional objects and advantages of the disclosed embodiments will beset forth in part in the description that follows, and in part will beapparent from the description, or may be learned by practice of thedisclosed embodiments.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the disclosed embodiments, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this specification, illustrate various exemplary embodiments andtogether with the description, serve to explain the principles of thedisclosed embodiments.

FIG. 1 depicts an exemplary block diagram of a system fortransition-coded media, measuring engagement of transition-coded media,and distribution of components of transition-coded media, according toone or more embodiments.

FIGS. 2-7 depict flowcharts for transition-coded media, according to oneor more embodiments.

FIGS. 8-13 depict flowcharts for measuring engagement oftransition-coded media, according to one or more embodiments.

FIGS. 14-18 depict flowcharts for distribution of components oftransition-coded media, according to one or more embodiments.

FIG. 19 depicts an example system that may execute techniques presentedherein.

DETAILED DESCRIPTION OF EMBODIMENTS

Various embodiments of the present disclosure relate generally tosystems and methods for media.

The terminology used below may be interpreted in its broadest reasonablemanner, even though it is being used in conjunction with a detaileddescription of certain specific examples of the present disclosure.Indeed, certain terms may even be emphasized below; however, anyterminology intended to be interpreted in any restricted manner will beovertly and specifically defined as such in this Detailed Descriptionsection. Both the foregoing general description and the followingdetailed description are exemplary and explanatory only and are notrestrictive of the features, as claimed.

In this disclosure, the term “based on” means “based at least in parton.” The singular forms “a,” “an,” and “the” include plural referentsunless the context dictates otherwise. The term “exemplary” is used inthe sense of “example” rather than “ideal.” The term “or” is meant to beinclusive and means either, any, several, or all of the listed items.The terms “comprises,” “comprising,” “includes,” “including,” or othervariations thereof, are intended to cover a non-exclusive inclusion suchthat a process, method, or product that comprises a list of elementsdoes not necessarily include only those elements, but may include otherelements not expressly listed or inherent to such a process, method,article, or apparatus. Relative terms, such as, “substantially” and“generally,” are used to indicate a possible variation of ±10% of astated or understood value.

In general, the present disclosure is directed to systems and methodsfor transition-coded media, measuring engagement of transition-codedmedia, and distribution of components of transition-coded media. Forinstance, a user device of the present disclosure may enable a user tointeract with a media file that includes tags that may instruct the userdevice and/or one or more secondary devices to perform functions toimmerse a user in the content of a media file. Therefore, a user'sexperience of a transition-coded media may enhance content consumptionof a media file.

Moreover, the user device and the one or more secondary devices, inaccordance with the tags of the media file, may track a user's state and(1) perform actions on a conditional basis of the user's state; (2)change the content of the media file based on the user's state; and/or(3) provide reaction data to a server. Therefore, the user device mayenable user-specific experiences of transition-coded media and enableauthors (e.g., authors of tags) to track how effective the media fileand tags are causing users to react.

Additionally, the user device may track the user's state to determinehow engaged the user is with the content (in general andcharacters/object/sections of media files in particular). Therefore,authors may be informed of how effective content of the media fileand/or a component (e.g., a particular tag) is at causing change in auser's state.

Lastly, a server may host a marketplace to gather and publish data forcomponents and distribute components to authors and end users. Thepublished data may indicate an average user reaction to a media fileand/or tag. The distributed components may enable authors to includetags into their transition-coded media to enhance a user's experience.Therefore, transition-coded media may be developed quickly and trackedfor effectiveness.

FIG. 1 depicts an exemplary block diagram of a system 100 fortransition-coded media, measuring engagement of transition-coded media,and distribution of components of transition-coded media, according toone or more embodiments. The system 100 may include one or more userdevice(s) 105, a network 110, one or more server(s) 115, a plurality ofdevices 120 (including 120A-120E), one or more user location(s) 125,and/or one or more local network(s) 130. Hereafter, while the one ormore server(s) 115 and the plurality of devices 120A-120E may interactwith one or a plurality of the one or more user devices 105, thisdescription will refer to the one or more user devices 105 as “the userdevice 105,” so as to simplify the description of the concepts describedherein. One of skill in the art would recognize that the one or moreserver(s) 115 may configure the one or more user devices 105 as toexperience different functionalities and/or have access to differentinformation (e.g., based on user settings, subscriptions, purchases,etc.). Additionally, while the functionality carried out by the one ormore server(s) 115, the plurality of devices 120A-120E and/or the userdevice 105 are discussed herein separately, in practice these featuresmay be executed on more or fewer devices.

The user device 105, the one or more server(s) 115, and the plurality ofdevices 120A-120E may be connected via the network 110 and/or via theone or more local network(s) 130, using one or more standardcommunication protocols. The network 110 may be one or a combination ofa wide area network (e.g., the Internet), an enterprise network, orother network. The user device 105, the one or more server(s) 115, andthe plurality of devices 120A-120E may transmit and receive messagesfrom each other across the network 110 and/or the one or more localnetwork(s) 130.

The user device 105 may include a display/user interface (UI) 105A, aprocessor 1058, a memory 105C, and/or a network interface 105D. The userdevice 105 may be a computer, a cell phone, a tablet, an electronicreader, a television, an automated teller machine (ATM), etc. The userdevice 105 may execute, by the processor 105B, an operating system (O/S)and at least one application (each stored in memory 105C). Theapplication may be a personal computer (e.g., desktop or laptop) programthat enables a user to interact with media files; a browser program(e.g., on a personal computer) that enables a user to interact withmedia files (e.g., by streaming of the media file); an application on anAutomated Teller Machine (ATM) that enables a user to interact withmedia files (e.g., ads or other content associated with the ATM, such asbanking information); an application on a television that enables a userto interact with media files; or a mobile application program (which mayalso be a browser program in a mobile O/S) that enables a user tointeract with media files (streaming or otherwise). The application maygenerate one or more graphic user interfaces (GUIs) based oninstructions/information stored in the memory 105C,instructions/information received from the one or more server(s) 115,and/or instructions/information received from the plurality of devices120A-120E. For instance, the GUIs might be application GUIs for theapplication executed based on XML and Android programming languages orObjective-C/Swift, but one skilled in the art would recognize that thismay be accomplished by other methods, such as webpages executed based onHTML, CSS, and/or scripts, such as JavaScript. The display/UI 105A maybe a touch screen or a display with other input systems (e.g., mouse,keyboard, etc.). The network interface 105D may be a TCP/IP networkinterface for, e.g., Ethernet or wireless communications with thenetwork 110. The processor 1058, while executing the application, mayreceive user inputs from the display/UI 105A, and perform actions orfunctions in accordance with the application.

The application, executed by the processor 1058 of the user device 105,may (1) display a graphical user interface (GUI) on the display/UI 105Afor a transition-coded media process and/or (2) execute thetransition-coded media process without displaying a GUI on thedisplay/UI 105A. The application may store user data on the user device105. The user data may include user identifiers (ID) for users of theuser device 105; baseline engagement for each of the user IDs (ifestablished); reaction dataset(s) (discussed below) for each of the userIDs; and/or user profiles for each of the user IDs. The user profilesincluding at least user preference data (discussed below) for the userID of the user profile. The user data may also include user settingsthat indicate which dynamic media environment enabled devices and/or oneor more actions are allowable.

The transition-coded media process may enable the user device 105 toobtain a media file and enable a user to interact with the media file,as discussed in more detail below. Generally, transition-coded mediafiles may include a media element and one or more tags.

The media element may be content to be displayed, content to be outputas audio, etc. on the user device 105. The one or more tags may beassociated to one or more progress points in the media element. Theprogress points may be indexed to a time track (if a data stream mediaelement) or to specific portions of content (if a static media element).The specific portions of content may include sections, chapters,paragraphs, sentences, words or phrases; webpages or portions thereof,such as text, images, or other HTML elements; application features, suchas text, images or other media; and/or regions of an image (left, right,top, bottom, etc.).

The tags may include one or more tag devices and, for each of the one ormore tag devices, one or more actions. The tag devices may be the userdevice 105, devices 120 that have general hardware (device 120 has,e.g., speaker or display), and/or devices 120 that have specifichardware (device 120 has specific hardware to perform specific action).As an example, a tag may indicate a tag device that has specifichardware corresponding to an ice maker of a refrigerator, and an actionmight be to cycle the ice maker for a defined period of time.

Generally, a tag may be one or a combination of: an action tag, areaction tag, a character tag, an object tag, and/or baseline engagementtags. The action tag may cause one or more devices from among the userdevice 105 and/or the devices 120A-120E to perform a defined function atparticular point in time (e.g., instantaneous or at a point in thefuture), for a particular length of time (e.g., a defined number ofseconds or repetitions). The reaction tag (“<reaction>” described infurther detail below) may cause one or more devices from among the userdevice 105 and/or the devices 120A-120E to track a user's state and/orreaction, as discussed in more detail below in FIGS. 8-13. Additionally,the reaction tag may be associated (e.g., paired together at a sameprogress point) with any other tag to track a user's state and/orreaction to the other tag. The character tag may indicate a character ispresent in the story/movie/etc. to track a user's reaction to thecharacter and/or the character actions. The object tag may indicate anobject is present in the story/movie/etc. to track a user's reactionto/engagement with the object. The baseline engagement tags may indicateportion(s) of a media element that is acceptable to perform a baselineengagement process (discussed below).

Below is a non-exhaustive list of tags (in the form of extensible markuplanguage (XML), but one of skill in the art would recognize that anymarkup language or data structure may be used indicate the one or moretag devices and, for each of the one or more tag devices, the one ormore actions. Below is a non-exhaustive list of general function tags:

<media start> Informs the user device 105 that tags exist;

<media end> Tells the user device 105 that the media transitions arefinished;

<environmental controls detect> The user device 105 discovers availableenvironmental controls native to the device being used, such as cameras,flash, backlighting, battery/heat, speakers, pressure;

<environmental controls report> Provides a listing of detected controlsto the user device 105;

<media environmental default difference> Provides a listing of settingsnot available to the transition-coded media default due to devicelimitations;

<user specific environmental controls> User-initiated connections to theone or more of the plurality of devices 120A-120E (such as HVAC, roomlighting, robotic massage chairs, room speakers, displays, other users'devices, etc.); and

<reset> Initiates a reset to default setting.

Below is a non-exhaustive list of action tags:

<device> Initiates control over any particular device that is part ofthe user device 105, such as camera, speakers, microphone, screen, etc.,and each of these would have their own functions;

<amplify> Initiates control over the magnitude of the device controlled,such as speakers become louder/quieter, etc.;

<percent> Initiates control over the default difference to adjust thedifference, such as if the default backlight is 20%, the <percent> tagmay adjust+/−1%;

<timing> Initiates control over when a device of the devices 120A-120Eactivates a function with sub-routines for “on” as # of seconds, “off”as immediate, and <increment> so that the device activation may ramp upover the time specified in “On=15 seconds;”

<programmed behavior> Initiates a cascade of pre-programmed controlsthat might activate multiple devices of the devices 120A-120E as apackage, such as lights dim and AC reduces room temperature by 2degrees;

<reaction> Initiates capture of the user's state/reaction during thereaction tag;

<link> Initiates a hyperlink to content; and

<prompt> Requests a user input as defined by the author, such as. “Likeon Social Media”, “More Info”, “Share”, “Survey.”

The one or more actions may be defined by an author of the tag. Anon-exhaustive list of actions may include: output a sound (custom ordefault for device 120); display image or video; turn on/off/changelighting; activate hardware (e.g., ice maker); etc. Generally, furtheractions may include: changing backlight or some other lighting of theuser device 105 or one of the devices 120 (referred to as action devicesdiscussed below); cause an onscreen alert to display on user device 105or action device; cause vibratory alerts and other haptic feedback ofuser device 105 or action device; cause audio players to output definedsounds; change environment conditions (HVAC fan and/or temperature) bycontrolling a HVAC system; causing environmental lighting controls toadjust; muscle stim; 3D movement; virtual reality; or augmented reality,etc.

The tags may also include conditions (e.g., indicated by conditionindicators of: true or false). The conditions may be arbitrarily complexas defined by an author. A condition may include a logical expression.The logical expression may include variables and logical operators(following an order of operations). The variables may correspond to theprogress of the media element, bits of information of a user state(e.g., heart rate, alertness, etc., as defined by the author of the tag)from the tracking data, content of the media element (e.g., user'saccount status on a mobile application screen), and/or other contextualinformation (e.g., user's account status, a credit score, birthday,etc.). The logical operators may include “and,” “or” (exclusive orinclusive), “not,” “nand,” and “nor,” etc. If the condition returnssatisfied, the tag may be executed.

The one or more server(s) 115 may include a display/UI 115A, a processor1156, a memory 115C, and/or a network interface 115D. The one or moreserver(s) 115 may be a computer, system of computers (e.g., rackserver(s)), and/or or a cloud service computer system. Hereafter, whilethe one or more server(s) 115 may interact with one or a plurality ofthe one or more user devices 105 and/or one or more of the plurality ofdevices 120A-120E, this description will refer to the one or moreserver(s) 115 as “the server 115,” so as to simplify the description ofthe concepts described herein. The server 115 may execute, by theprocessor 1156, an operating system (O/S) and at least one instance of aserver transition-coded media program (each stored in memory 115C). Theserver 115 may store or have access to server transition-coded mediainformation (e.g., hosted on a third-party server). The display/UI 115Amay be a touch screen or a display with other input systems (e.g.,mouse, keyboard, etc.) for an operator of the server 115 to control thefunctions of the server 115 (e.g., update the server transition-codedmedia program and/or the server transition-coded media information). Thenetwork interface 115D may be a TCP/IP network interface for, e.g.,Ethernet or wireless communications with the network 110.

The server transition-coded media program, executed by the processor1156 on the server 115, may (1) coordinate with the plurality of devices120A-120E to configure/update the plurality of devices 120A-120E; (2)coordinate with the user device 105 to interact with media files; and/or(3) coordinate with the user device 105 to create media files (when theuser device 105 is an authoring device). Further details of the servertransition-coded media program are discussed below. The servertransition-coded media information may include analytics data based onuser reactions to tags. The analytics data may include media engagementdata and/or user preference data.

The media engagement data may include media file profiles andserver-user profiles. Generally, the media engagement data includes aplurality of reaction datasets for all of the components hosted on theservice. The media file profiles include reaction datasets for allcomponents hosted on the service (e.g., for each component ID), storedin association with a media element. The media file profiles may providehigh level user-anonymous data to indicate effectiveness of media filesand components (e.g., tags, tag devices, and actions) to implement adynamic media environment, and describes the component in a descriptionor meta-data. The server-user profiles may include reaction datasets forall users of the service (e.g., for each user ID).

Each reaction dataset may include a user ID, a component ID, and/orreaction data. The reaction data may include a progress of the mediaelement, a user state, and/or a user profile from a reaction messagetransmitted in response to a reaction trigger event of a reaction tag.The user ID may correspond to a user device 105 that transmitted thereaction message. The component ID may correspond to the reaction tag ora tag associated with the reaction tag (hereinafter, “reaction tag” forease of reference). The user profile may store an average of thenumerical values of the user state, an average user mode, and/or anaverage user mood (“baseline data”), user preferences in the userpreference data, and detected deviations from the baseline data.

The media engagement data may also include averages and/or scores for acomponent ID. The averages and/or scores may be averages and/or scoresof the reaction data for the component ID, such as an average/score ofusers' states, average/score of user profiles, and/or average/score of adifference between users' states and baseline states of the media fileprofiles. The averages and/or scores may indicate an effectiveness of areaction tag or the tag associated with a reaction tag (such a charactertag/object tag). The average/scores may be determined for each data typeof the reaction data (heart rates, movement data, etc.). The server 115may be determine the average of a data type based on an arithmetic meanfunction, geometric mean, or harmonic mean. The scores may be determinedbased on a scoring algorithm. The scoring algorithm may bemulti-variable function or machine learning model that takes theaverages and/or the plurality of datasets for a component ID as aninput.

For each user ID, the user preference data may indicate a user'spreference for types of media elements (e.g., visual over audio), auser's preference for sections of media elements (e.g., start, middle,end, particular pages/scenes/refrains, etc.), and/or a user's preferencefor particular content of media elements (e.g., a person/character of astory or video, objects, etc.), based on reaction data for the user ID.The server 115 may determine the user preference data based on userreactions to different types of media, different sections of a mediaelement, and/or tags of the media element (e.g., the media element mayinclude character tags/object tags). For instance, the server 115 mayanalyze user reactions while interacting with a media element usingmachine learning models to determine the user's preferences.

The server 115 may host a service, discussed in more detail in FIGS.14-18 below. Generally, the service may be a market place to (1) gatherdata on components and (2) distribute components to implementtransition-coded media. For instance, the service may enable access toone or combinations of: engagement index information for a type ofmedia; engagement profiles for effects and/or dynamic processes;registration information, and/or modification information (e.g.,“marketplace data”). The service may enable distribution of themarketplace data so that authors (e.g., some users of user devices 105)may generate transition-coded media with effective components by usingframeworks of already defined tags. Moreover, authors of components(tags, action/tracking devices (such as devices 120A-120E), and actions)may review engagement index information and modify engagement profilesto determine effectiveness of components and media paths.

The engagement index information for the type of media may includesummaries of media file profiles indicating responses of one or moreuser(s) to deployed transition-coded media and tags, action/trackingdevices, and actions thereof. In this manner, data on how effective atag, action/tracking devices, and actions may be published. Authors oftransition-coded media may use the published data to quickly find acomponent by searching for a type of a component using the user device105. As an example, the author may search for a particular type ofaction tag. The server 115 may transmit a search result including thepublished data and a link to the engagement profiles for components thatmatch a search parameter. The author may then navigate, via the userdevice 105, to the engagement profile of a component; request, via theuser device 105, a framework for the searched for component; receive theframework for the component (e.g., a tag) and edit the transition-codedmedia to include the framework for the component. In this manner, theauthor may make informed decisions about which components to use in theauthoring process of a transition-coded media, and quickly includecorresponding components in the transition-coded media.

The engagement profiles for the effects and/or the dynamic processes mayinclude one or more action tag processes to generate specific effectsand/or specific dynamic processes for a transition-coded media.Generally, the one or more action tag processes may provide a frameworkto insert a tag into a transition-coded media so that thetransition-coded media includes a tag with one or more tag devices and,for each of the one or more tag devices, one or more actions. Theframework may or may not be editable by the author. The author of theengagement profile selected by a secondary author may receive aconsideration (e.g., money or benefits, such as a reduction in asubscription fee for the service), or the author may make the engagementprofile available freely without consideration. The deployed tag basedon the framework in a transition-coded media may report user reactiondata for the deployed tag as discussed below in FIG. 18; moreover, theuser reaction data for the deployed tag may also report user reactiondata for an original tag that was the basis of the engagement profile,thereby increasing data available to indicate effectiveness of acorresponding engagement index information.

The registration information may include information to register adevice to interact with a transition-coded media. Once a device isregistered (e.g., a tracking and/or action device such as devices120A-120E), user devices such as the user device 105 may execute asearch process for dynamic media environment enabled devices andinteract with the registered device.

The modification information may include information to modify a deviceto be able to interact with the transition-coded media. For instance,the modification information may include a management application andone or more function applications. The management application maycontrol the modified device to handle processing of messages to and fromuser devices 105/the server 115. Meanwhile, the one or more functionapplications may control the device to execute a tracking functionand/or an action function (discussed below). End users of the servicemay download the modification information and update devices thatpreviously did not provide a tracking function and/or an action functionin the dynamic media environment so that the devices may provide atracking function and/or an action function.

The plurality of devices 120A-120E may be tracking devices and/or actiondevices, depending on whether the plurality of devices 120A-120E have atracking function and/or an action function enabled. The plurality ofdevices 120A-120E may be any device that is: (1) enabled to communicatewith a user device 105 directly or indirectly, and (2) includes hardwareto execute a tracking function and/or an action function (stored on amemory of the device 120 and executed on a processor of the device 120).For instance, the plurality of devices 120A-120E may include one orcombinations of: a cell phone, a smart phone, an AR goggle, a VR goggle,a tablet, a laptop, a desktop computer, Internet of things (IoT) devices(e.g., such as a refrigerator, an oven, a dishwasher, any other kitchenappliance, a washer, a dryer, a HVAC system, a security system, aconnected voice controller, a connected speaker, a connected doorbell, aconnect light, and/or a connected switch), and/or an audio-visualcomputer equipment. The plurality of devices 120A-120E may also includeone or more of the one or more user device(s) 105 (when set as atracking device or action device).

The plurality of devices 120A-120E may be tracking devices if theplurality of devices 120A-120E have hardware to execute a trackingfunction and the tracking function is enabled. The tracking function mayuse the hardware to obtain tracking data. The tracking data may includedifferent types of tracking data. The hardware to obtain tracking datamay include: a microphone to record sound; a camera to record images,such as pictures or video, with an image/video analyzer (e.g., a machinelearning model) to detect persons and their mode and/or mood; hapticsensors to sense user touch or proximity; health sensors to measure bodystates (e.g., a heart rate monitor, etc.); accelerometer/gyroscope/GPSto track movement and/or orientation of the user/user device 105; and/orthermometer to detect ambient temperature or user temperature. Thetracking devices may process the tracking data to determine a user stateand/or environment data. The user state may include values for usercharacteristics based on the tracking data. The values may be a currentvalue (e.g., based on most recently received information) or a datastream of values for the user characteristics indexed to a time of asystem clock of the user device 105. The tracking device(s) and the userdevice 105 may be time synced so the values may be correctly indexed towhen the tracking device obtained the bit of information. The usercharacteristics may include (1) user mode indicator indicating the useris one or more of talking, singing, sitting, standing, moving, at rest(e.g., reclining), etc.; (2) user mood (happy, sad, angry, restless,impatient, etc.); and/or (3) user physiological state (e.g., heart rate,body temperature, perspiration, etc.). The environment data may includeambient noises, lighting, and/or temperature.

Moreover, for static media elements (e.g., e-books, webpages, staticcontent on features of the application), the user device 105 and/ordevices 120 may track a user's progress through the static mediaelement. For instance, the user device 105 and/or devices 120 may trackthe user's progress through the static media element by tracking userinputs (e.g., into to turn page of e-book, webpage navigation, featurenavigation of the application, inputs on sections (e.g., area ofdisplay) of the content, etc.) and/or tracking a user's attention (e.g.,by eye tracking on the content).

The plurality of devices 120A-120E may be action devices if theplurality of devices 120A-120E have hardware to execute an actionfunction and the action function is enabled. The action function may usethe hardware to execute activation mechanisms. The hardware to executeactivation mechanisms may be any electrical and/or mechanical system ofthe device 120 that can output sound, vibration, light, images, video,scents, taste, or any combination thereof.

The one or more user location(s) 125 may geographic locations associatedwith a user device 105 and/or one or more of the plurality of devices120A-120E. For instance, a user device 105 may be associated with a userlocation 125 that is a home, work, vehicle, etc. of a user of the userdevice 105. The user location 125 may have a data structure thatincludes associated user device information and associated deviceinformation. The associated user device information may indicate userdevices 105 associated with the user location 125, such as users thatare a part of a family (or any group of users) that have one or moreuser devices 105. The associated device information may indicate one ormore of the plurality of devices 120A-120E associated with the userlocation 125, such as devices 120C, 120D, and 120E of user location 125,as shown in FIG. 1. The devices 120C, 120D, and 120E may be associatedwith the user location 125 by being on a same local network 130 of theuser location 125 and/or by being within a threshold distance of a userdevice 105. Further details of one or more user locations 125 arediscussed below.

FIGS. 2-7 depict flowcharts for transition-coded media, according to oneor more embodiments. FIG. 2 may depict a flowchart 200 for enabling auser to interact with a media file. Generally, the user device 105 andnone, one, or more of plurality of devices 120A-120E acting as trackingdevices and/or action devices may cooperate to enable a user to interactwith a media file. The user device 105 may execute the application tointeract with the media file, and the tracking devices and/or actiondevices may execute tracking functions and/or action functions,respectfully.

The user device 105 may start the process of the flowchart 200 byobtaining a media file (block 205). For instance, the user device 105may obtain the media file by downloading or streaming the media filefrom the server 115 or another server or user device. For instance, theuser device 105 may download or stream the media file in response to auser input selecting a media file, or the user device 105 may downloadthe media file in response to an instruction from, e.g., the server 115so as to display an advertisement or other content on a television,display, or ATM.

The user device 105 may then proceed to determine whether the media fileincludes transition-coded media (block 210). For instance, the userdevice 105 may obtain the media file; open/parse the media file;determine whether media file includes a transition-coded mediaindicator; and, if so, determine the media file includestransition-coded media. The transition-coded media indicator mayindicate the media file includes a media element and one or more tagsfor the media element.

In response to determining the media file does not includetransition-coded media (block 210: No), the user device 105 may proceedto execute the media file without a dynamic media environment (block230). For instance, the user device 105 may play music, play a video,display a photo, display an advertisement, etc. without the dynamicmedia environment. In response to determining the media file includestransition-coded media (block 210: Yes), the user device 105 may thenproceed to establish the dynamic media environment (block 215). Forinstance, the user device 105 may establish the dynamic mediaenvironment as discussed below with reference to FIG. 4.

The user device 105 may then proceed to detect whether a user isinteracting with the transition-coded media (block 220). For instance,the user device 105 may detect a user is interacting with thetransition-coded media by determining (1) the user is using theapplication (in the case the content of the application is consideredtransition-coded media); (2) the user is using the application and theuser selected the transition-coded media (e.g., directly by a user inputselecting the transition-coded media or as a part of a playlist or otherauto-start process); and/or (3) the user is using the application andthe user is traversing (e.g., by turning pages of an e-book by userinputs, scrolling through content by user inputs, listening/viewingcontent, etc.) of the transition-coded media.

In response to not detecting the user is interacting with thetransition-coded media (block 220: No), the user device 105 may proceedto return to detect whether the user is interacting with thetransition-coded media (block 220). In response to detecting the user isinteracting with the transition-coded media (block 220: Yes), the userdevice 105 may then proceed to execute the transition-coded media withthe dynamic media environment (block 230). For instance, the user device105 may execute the transition-coded media with the dynamic mediaenvironment, as discussed below in FIG. 3.

FIG. 3 depicts a flowchart 300 for executing the transition-coded mediawith the dynamic media environment.

The user device 105 may start the process of the flowchart 300 bystarting to track a progress of a media element and a user state (block305). For instance, the user device 105 may track a progress of a mediaelement and a user state as discussed below with reference to FIG. 5.

The user device 105 may then proceed to determine whether a triggerevent occurs (block 310). For instance, the user device 105 maydetermine whether a trigger event occurs as discussed below withreference to FIG. 6.

In response to determining a trigger event does not occur (block 310:No), the user device 105 may proceed to return to determining whether atrigger event occurs (block 310). In response to determining a triggerevent occurs (block 310: Yes), the user device 105 may then proceed todetermine a transition action (block 315). For instance, the user device105 may determine a transition action as discussed below with referenceto FIG. 7. Briefly, the user device 105 may determine the transitionaction based on a selection of available dynamic media environmentenabled devices that match tag devices of the tag, and a selection ofactions of the matching devices that match actions of the tag.

The user device 105 may then proceed to perform the transition action(block 320). For instance, the user device 105 may perform thetransition action by generating instructions to perform the transitionaction, and either (1) executing the instructions on the user device 105and/or (2) transmitting an instruction message to the selected matchingdevices. The instruction message may include the instructions to executethe transition action and/or time to execute the transition action.

The user device 105 may then proceed to determine whether thetransition-encoded media has ended (block 325). In response todetermining the transition-encoded media has not ended (block 325: No),the user device 105 may return to determine whether a trigger eventoccurs (block 310). In response to determining the transition-encodedmedia has ended (block 325: Yes), the user device 105 may stop executingthe media file with the dynamic media environment (block 330).

FIG. 4 depicts a flowchart 400 for establishing a dynamic mediaenvironment. The user device 105 may start the process of the flowchart400 by determining a location of dynamic media environment (block 405).For instance, the user device 105 may determine the location of thedynamic media environment by: (1) determining which network the userdevice 105 is connected to and/or (2) obtaining a geographic position ofthe user device 105 based on a GPS signal.

The user device 105 may then proceed to determine whether the locationis a known location (block 410). For instance, the user device 105 maydetermine whether the location is a known location by: obtaining knownlocation information from, e.g., a memory (such as memory 105C) or fromthe server 115; extracting known networks and/or known positions fromthe known location information; determining whether the currentlyconnected network is among the known networks (e.g., a local network130) or whether the current geographic position is within a thresholddistance to a known location (e.g., within 2, 5, 50 feet, etc.). Theknown locations may associated with devices 120. As an example, asdiscussed below, the user device 105 may be in near proximity (e.g.,within the threshold distance) of a device 120 (such as a television orATM) and the user device 105 may prompt the user of the user device 105if the user does not show interest in content being displayed by thedevice 120.

In response to determining the location is not a known location (block410: No), the user device 105 may proceed to execute a search processfor dynamic media environment enabled devices (block 425). For instance,to execute a search process for dynamic media environment enableddevices, the user device 105 may transmit query messages on a localnetwork 130 or directly to nearby devices 120 to discover devices 120;receive response messages from the devices 120 over the local network130 or directly from the devices 120; extract tracking and/or actionfunction information from the response messages for the devices 120 thatresponded; store the tracking and/or action function information aslocation data in association with the location and as a new location;and update the known location information with the new location (e.g.,new network or new geographic position). In the future, this locationmay be determined as a known location and the search process may beomitted thereby reducing processing time and processing resources. Thetracking and/or action function information may indicate trackingfunctions and/or action functions the devices 120 are capable of in thedynamic media environment. The user device 105 may then proceed toinitialize dynamic media environment enabled devices (block 430), asdiscussed below.

In response to determining the location is a known location (block 410:Yes), the user device 105 may then proceed to obtain location data forthe location (block 415). For instance, the user device 105 may retrievethe location data from the memory (e.g., memory 105C) by matching thecurrent network or current position to a network or position from amonga plurality of location data (each being associated with a network orposition). The plurality of location data may include a timer for eachlocation data that indicates how long ago a search process was executedfor the location data.

The user device 105 may then proceed to determine whether the locationdata has expired (block 420). For instance, the user device 105 maydetermine whether the timer for the location data has exceeded anexpiration threshold, which may be adjustable based on characteristicsof the location (e.g., shorter time periods for locations thatexperience higher changes in devices 120 associated with the location,longer periods for locations that experience lower changes in devices120 associated with the location).

In response to determining the location data has expired (block 420:Yes), the user device may execute the search process for dynamic mediaenvironment enabled devices (block 425). In this case, the searchprocess may update the location data with the tracking and/or actionfunction information received from devices 120 currently at thelocation. Therefore, the user device 105 may have a currentunderstanding of the tracking functions and/or action functions for thedevices 120.

In response to determining the location data has not expired (block 420:No), the user device may initialize dynamic media environment enableddevices (block 430). To initialize dynamic media environment enableddevices, the user device 105 may transmit, to the devices 120 of thetracking and/or action function information, initialization instructionsto be prepared for potential further instructions from the user device105. For instance, the potential further instructions may includetracking instructions and/or instructions to execute an action, inaccordance with the transition-coded media. The devices 120, in responseto initialization instructions, may be in stand-by mode (with or withoutindicating a change in state) so as to timely execute tracking and/oractions, and may continue in stand-by mode until an end instruction isreceived (e.g., transmitted by the user device 105 when the user device105 stops executing the media file with the dynamic media environment).

FIG. 5 depicts a flowchart 500 for tracking a progress of a mediaelement and a user state for block 305 of FIG. 3. The user device 105may start the process of the flowchart 500 by sending instruction(s) totrack a user state to tracking device(s) (block 505). For instance, theuser device 105 may obtain the tracking and/or action functioninformation for the location; extract tracking devices and informationabout the tracking functions for the tracking devices from the trackingand/or action function information; and send instructions to one or moreof the tracking devices to perform tracking in accordance to thetracking functions of the devices 120. Note, the user device 105 mayalso be a tracking device, so the “sending” may include transmitting aninstruction from the application to a tracking function of the userdevice 105 to track the user state. The instruction may indicate anexpiration time for the tracking (e.g., for the next period of time totrack the user state) or may indicate that the devices 120 are to trackthe user state until an end tracking instruction is received (or an exittimer ends). The devices 120 may transmit tracking data continuously orperiodically (e.g., multiple times a second, a minute, etc.) to the userdevice 105.

The user device 105 may then proceed to determine whether tracking datahas been received (block 510). In response to determining tracking datahas not been received (block 510: No), the user device 105 may proceedto return to determine whether tracking data has been received (block510). In response to determining tracking data has been received (block510: Yes), the user device 105 may then proceed to process tracking datato obtain a current user state (block 515). For instance, the userdevice 105 may parse the tracking data to obtain bits of informationfrom the tracking data relevant to tags of the transition-coded mediaand/or store all tracking data for data analysis, as discussed below.For instance, if a tag indicates a heart rate of a user is an input to acondition of the tag, the user device 105 may parse out heart rateinformation from the tracking data; if the media element is static mediaelement, the user device may extract a user's progress through thestatic media element.

The user device 105 may then proceed to determine a type of mediaelement (block 520). For instance, the user device 105 may retrieve amedia type ID from the transition-coded media, and determine whether themedia element is a static media element or a data stream media element.A static media element may not be indexed to a time track, while datastream media element may be indexed to a time track. In response todetermining the type of media element is a static media element (block520: Static Media Element), the user device 105 may proceed to determinea progress of the media element based on the current user state (block525). For instance, the user device 105 may determine the progress ofthe media element by determining the user's progress through the staticmedia element from the tracking data. In response to determining thetype of media element is a data stream media element (block 520: DataStream Media Element), the user device 105 may then proceed to determinethe progress of media element based on time (block 530). For instance,to determine the progress of a media element based on time, the userdevice 105 may determine the progress of the media element bydetermining a play time of the media element or current time of a timer.

The user device 105 may then proceed to return the progress of mediaelement and the current user state (block 535). For instance, the userdevice 105 may then return to block 305 and continue to block 310.

FIG. 6 depicts a flowchart 600 for determining whether a trigger eventoccurs for block 310 of FIG. 3. The user device 105 may start theprocess of the flowchart 600 by obtaining the current user state andprogress of media element (block 605), as discussed above.

The user device 105 may then proceed to determine whether a progresspoint of a tag has been reached (block 610). For instance, the userdevice 105 may determine whether a progress point of a tag has beenreached based on the progress of the media element. Specifically, theuser device 105 may compare the progress of media element to progresspoints of the tags; and determine whether the indexed progress point(for a data stream media element) for a tag has been reached (and hasnot been passed over by a threshold amount of time (e.g., a second, amicro-second, etc.), or determine whether specific portions of contentassociated with a tag (for a static media element) is currently beingviewed by a user.

In response to determining a progress point of a tag has not beenreached (block 610: No), the user device 105 may proceed to obtain thecurrent user state and the progress of media element (block 605). Inresponse to determining a progress point of a tag has been reached(block 610: Yes), the user device 105 may then proceed to determine atrigger event for the tag (block 615). For instance, the user device 105may determine the trigger event corresponds to the tag and pass thetrigger event and tag to block 310 of FIG. 3 so that the user device 105may proceed to block 315 of FIG. 3.

FIG. 7 depicts a flowchart 700 for determining a transition action forblock 315 of FIG. 3. As discussed in FIG. 6, the trigger event and tagmay be returned so that the user device 105 may determine a transitionaction. The user device 105 may start the process of the flowchart 700by determining whether the tag includes a condition (block 705). Forinstance, the user device 105 may extract a condition indicator from thetag, and determine whether the condition indicator is true.

In response to determining the tag does not include a condition (block705: No), the user device 105 may proceed to determine the transitionaction based on the tag (block 725). For instance, the user device 105may extract information for one or more tag devices from the tag;compare the information for the one or more tag devices to informationfor available dynamic media environment enabled devices (e.g., foraction devices from the tracking and/or action function information);determine matching devices based on a comparison result of the comparingthe information for the one or more tag devices to information for theavailable dynamic media environment enabled devices (e.g., actiondevices such as the user device 105, devices 120 that have generalhardware, and/or devices 120 that have specific hardware); and determinethe transition action based on the matching devices. To determine thetransition action based on the matching devices, the user device 105 mayselect one or the matching devices; and select an action of one or moreactions of the tag that matches the actions possible for the actiondevice. The user device 105 may then proceed to return the transitionaction (block 730). For instance, the user device 105 may then return toblock 315 and continue to block 320.

In response to determining the tag does include a condition (block 705:Yes), the user device 105 may then proceed to determine whether thecondition is satisfied (block 710). For instance, the user device mayextract a logical expression from the condition of the tag; obtain bitsof information from the tracking data (in accordance with the tag)and/or progress of the media element; set the bits of information fromthe tracking data and/or progress of the media element as the variablesof the logical expression; and determine whether the logical expressionis satisfied.

In response to determining the condition is not satisfied (block 710:No), the user device 105 may proceed to store a corresponding portion ofthe tracking data (block 735). For instance, the user device 105 maystore in the user profile the corresponding portion to determine why thecondition was not satisfied for feedback, as discussed below.

In response to determining the condition is satisfied (block 710: Yes),the user device 105 may then proceed to determine the transition actionbased on the tag (block 715). For instance, the user device 105 mayproceed as in block 725 discussed above. The user device 105 may thenproceed to return the transition action (block 720). For instance, theuser device 105 may then return to block 315 and continue to block 320.

Examples of the systems and methods of this aspect of the disclosure:

Example 1: A User is listening to music on their phone. Because themusic is transition-coded media, during solos the music uses the phoneto detect indicators that the user is enjoying the solo, such as arethey head banging during guitar riffs, are they signing along with thesinger, and are they tapping to the beat? If yes, the music file couldrespond with certain actions: (1) enhance the sound of the guitar riff,or message the user a video of the guitar player playing that song; (2)the words of the song could display on their phone; (3) a story about orby the drummer describing what it's like to play that song is texted tothe user. The user could “like” that section and set a preference basedon available transition options.

Example 2: A user is reading a book on an e-reader. During scenesinvolving their favorite character, the user may grip the edges of theire-reader more tightly. The book may be transition-coded media and it mayinclude a reaction tag to look for such reactions around key plot pointswith the story's characters. Detecting this for that character,backstory content may become available to the reader.

Example 3: A user has enabled all transitions for their movie night. Theuser lives in an integrated home having smart application link(s) totheir HVAC, lighting, and entertainment center. The transition-codedmedia may: automatically dim the room's lighting by 10% at a firstprogress point and automatically lower the room's temperature by 2degrees by instructing the HVAC system. A camera of their smart-TV,acting as a tracking device, may notice that the user is on the edge oftheir seat during a key battle scene, and a reaction tag may increasethe special effects' sounds by 10%. Another action tag could (e.g.,timed with an alien stalking a character) instruct a speaker in thekitchen to play the sounds of wet feet slapping against cold metal.

Example 4: A user has finished their transition-coded media and atransition summary may display showing things, such as: user seemed toenjoy the drum beat of this song; you accessed 10 of 56 transitions inthis media; character A was your favorite character to character Z asyour least favorite; and your favorite parts of this media were: X, Y,and Z.

Example 5: On a display adjacent to an ATM, information/images may bedisplayed (e.g., a card offering, highlighted by a celebrity or otherspokesperson). The ATM may have a camera, and the ATM could beintegrated into the display. During the user's interaction with the ATM,the camera (acting a tracking device) may detect that the user islooking at the information/images on the display; a reaction tag may betriggered by the user looking at the display causing a prompt to displayon the ATM. For instance, the prompt may ask if the user would like moreinformation.

Example 6: A user may be using their smartphone at the ATM of Example 5.In this example, instead of the reaction tag causing a prompt to displayon the ATM (as the user device 105 executing transition-coded media),the reaction tag may cause a text message to be sent to the user'ssmartphone to ask if the user would like more information. The user mayclick a link in the text message to access a video of interactivecontent (which is transition-coded media as well, so the smartphone isalso a user device 105). During the video of interactive content (astransition-coded media), the smartphone may track the user (user touchpressure increases when the celebrity or spokesperson talks about theirpersonal use of the card; user increases volume during instructions(maybe they want to apply but are having a hard time?); user's eyes keepdarting between their text messages and the content (maybe they arebusy?)); a reaction tag may have a condition for if a user's mode isdistracted, change media path to display video segment of the celebritysaying, “I sense you're busy. If you′d like, we can save this locationfor viewing at a later time?”

Example 7: A user is using a mobile application (e.g., a bankingapplication) and the mobile application may be a transition-coded media.A camera may detect eyes or shapes around the user, and in response to areaction tag may engage a privacy filter screen over the mobileapplication. A reaction tag may check a status of a user's account(e.g., by examining data on the screen of the mobile application and/oraccessing a user's account) and, when the account is in a good status,change the user experience of the mobile application (e.g., softenlighting, etc.). For handicap users, reaction tags may detect andtrigger screen experiences based on eye movement, hand tremble/pressure,etc.

Therefore, in this aspect of the disclosure, the systems and methods ofthe present disclosure may enhance content consumption of a media file.Besides for entertainment enhancements in content consumption, thesystems and methods of the present disclosure may also enable authors oftransition-coded media to provide user-specific experiences using tagswith conditions. Moreover, the systems and methods of the presentdisclosure may enable authors to interact with users as the usersinteract with the transition-coded media, such as by routing users toadditional materials (e.g., by links or prompts), providing an option toproceed at a different time (e.g., if distracted) or in a differentchannel (e.g., on mobile device instead of an ATM or TV), and/or providespecific feedback.

FIGS. 8-13 depict flowcharts for measuring engagement oftransition-coded media, according to one or more embodiments. FIG. 8depicts a flowchart 800 for measuring engagement of transition-codedmedia. Generally, the user device 105 and none, one, or more of theplurality of devices 120A-120E acting as tracking devices and/or actiondevices may cooperate to enable a user to interact with a media file,measure engagement of transition-coded media of the user, and performreaction actions.

The user device 105 may start the process of the flowchart 800 bystarting to track a progress of a media element and a user state (block805). For instance, the user device 105 may track the progress of amedia element and the user state in the same manner as discussed abovewith respect to the flowchart 500 of FIG. 5.

The user device 105 may then proceed to determine whether a baselineengagement has been established (block 810). For instance, the userdevice 105 may obtain a user ID for a user currently using the userdevice 105 (e.g., by determining which user is logged in to theapplication or currently active profile, etc.); obtain the user data;determine whether the user data includes a baseline engagement for theuser ID; if so, determine the baseline engagement has been established;and if not, determine the baseline engagement has not been established.

In response to determining the baseline engagement has not beenestablished (block 810: No), the user device 105 may proceed to performa baseline engagement process (block 815). For instance, the user device105 may perform a baseline engagement process as discussed below withreference to FIG. 9. A baseline engagement may include an average ofnumerical values of the user state, an average user mode, and/or anaverage user mood.

The user device 105 may then proceed to generate a user profile (block820). For instance, the user device 105 may extract the average of thenumerical values of the user state, the average user mode, and/or theaverage user mood (“baseline data”), and initiate a user profile withthe baseline data. The user profile may also store subsequent dataregarding user preferences in the user preference data, and deviationsfrom the baseline engagement (measured/obtained by the user device 105).

In response to determining the baseline engagement has been establishedor after the user profile has been generated (block 810: Yes), the userdevice 105 may then proceed to perform a reaction trigger process (block825). For instance, the user device 105 may perform the reaction triggerprocess as discussed below with reference to FIG. 10.

The user device 105 may then proceed to update the user profile (block830). For instance, the user device 105 may determine (or transmit tothe server 115 a request for the server 115 to determine) updatedpreferences for the user based on reactions obtained during the reactiontrigger process. Moreover, the user device 105 may update the userprofile with the deviations stored in the user profile. In this manner,the user device 105 may recognize change in the baseline, and ensure thebaseline reflects the user's baseline engagement state.

FIG. 9 depicts a flowchart 900 for performing a baseline engagementprocess as in block 815 of FIG. 8. The user device 105 may start theprocess of the flowchart 900 by obtaining a progress of a media elementand a user state (block 905), as obtained in block 805 of FIG. 8 above.The user device 105 may then proceed to determine whether there issufficient data (block 910). For instance, to determine whether there issufficient data, the user device 105 may determine whether a length oftime of tracking a user state is more than a threshold time (e.g., atleast thirty seconds); the user state is within a first range (e.g., ina calm/average/steady range); changes in the user state are below athreshold value; and/or tracking devices are gathering a defined minimumset of tracking data. The defined minimum may be set by an author of atransition-coded media. As an example, the defined minimum set oftracking data may include: heart rate of the user of user device 105,audio recording of the user of user device 105, and/or video recordingof the user of user device 105.

In response to determining there is not sufficient data (block 910: No),the user device 105 may proceed to return to obtain the progress of themedia element and the user state (block 905). In response to determiningthere is sufficient data (block 910: Yes), the user device 105 may thenproceed to determine the user baseline engagement (block 915).

For instance, the user device 105 may determine the baseline engagementbased on the user state and the progress of the media element.Specifically, the user device 105 may: determine a time window for thebaseline engagement; select a sub-set of the user state based on thetime window (e.g., select a portion of the data stream of values of theuser state in accordance with the time window); and determine thebaseline engagement based on the sub-set of the user state and theprogress of the media element.

To determine the baseline engagement based on the sub-set of the userstate and the progress of the media element, the user device 105 maydetermine whether the progress of the media element indicates a portionof the media element that is acceptable to perform the baselineengagement process (e.g., have not yet started interacting with themedia element, opening credits of a movie, starting portion of a song,etc.), or that any progress point is acceptable. The transition-codedmedia may indicate portions (or all) of the media element are acceptableby including baseline engagement tags. The baseline engagement tags mayinclude a start tag indicating a starting progress point and an end tagindicating an ending progress point, with any progress point in betweenindicated as acceptable to perform the baseline engagement process. Inresponse to determining the progress of the media element indicates aportion of the media element that is not acceptable, the user device 105may return to obtain the progress of the media element and the userstate (block 905).

In response to determining the progress of the media element indicates aportion of the media element that is acceptable, the user device 105 mayperform a time series analysis of the sub-set of the user state todetermine the baseline engagement. For instance, the time seriesanalysis of the sub-set of the user state may calculate a moving average(e.g., one or a combination of a simple, weighted, cumulative, orexponential moving average function) of numerical values of the userstate (e.g., a moving average of the user's heart rate) and determine anaverage user mode and user mood for the window of time. To determine theaverage user mode and user mood for the window of time, the user device105 may statistically analyze the user mode and user mood during thewindow of time to determine a most common (e.g., highest frequency)and/or most recent user mode and user mood. Specifically, the userdevice 105 may determine frequency of each mode/mood and a most recentmode/mood. If the highest frequency mode/mood is the same the as themost recent mode/mood (e.g., an agreed mode/mood), the user device 105may determine the agreed mode/mood as the average mode/mood. If there isa tie for highest frequency mode/mood (or the highest frequencymode/mood is within a threshold frequency of the second highestfrequency), the user device 105 may determine the most recent mode/moodas the average mode/mood. Generally, the user device 105 may determinethe average user mode and user mood based on tie-breaker rules andstatistics features of the sub-set of the user state for each mode/moodin the window of time. The statistics features of the sub-set of theuser state may be determined according to frequency-domain analysismethods and/or time-domain analysis methods. The tie-breaker rules maytake as inputs the statistics features and proceed to determine a usermode and mood, in a similar manner as discussed in the precedingparagraph. One skilled in art would understand that this rule-basedstatistical analysis may be varied in numerous ways.

The user device 105 may then proceed to return user baseline engagement(block 920). For instance, the user device 105 may then return to block815 and continue to block 820 of FIG. 8.

FIG. 10 depicts a flowchart 1000 for performing the reaction triggerprocess as discussed above with reference to block 825 of FIG. 8.Flowchart 1000 is similar to flowchart 300 of FIG. 3 discussed above,except that the tag is a reaction tag and, importantly, the determiningof the transition action (block 1015) further includes the flowchart ofFIG. 11.

FIG. 11 depicts a flowchart 1100 for determining a transition action fora reaction tag. The user device 105 may start the process of theflowchart 1100 by obtaining a reaction tag (block 1105). For instance,the user device 105 may have been passed the trigger event and reactiontag from block 615 of FIG. 6.

The user device 105 may then proceed to determine a type of feedback(block 1110). For instance, the user device 105 may extract a feedbacktype from the reaction tag; and determine whether the feedback type is amedia feedback type and/or an author feedback type.

In response to determining the type of feedback is the media feedbacktype (block 1110: Media Feedback Type), the user device 105 may proceedto determine a transition action as media feedback process (block 1115).In response to determining the type of feedback is the author feedbacktype (block 1110: Author Feedback Type), the user device 105 may thenproceed to determine a transition action as an author feedback process(block 1120).

The user device 105 may then proceed to return the transition action asthe media feedback process and/or the author feedback process so thatthe user device 105 may perform the transition action (block 1020).

Specifically, FIG. 12 depicts a flowchart 1200 for the media feedbackprocess. Meanwhile, FIG. 13 depicts a flowchart 1300 for the authorfeedback process.

Turning to flowchart 1200 for the media feedback process, the userdevice 105 may start the process of the flowchart 1200 by obtaining thereaction tag, a user profile, and a user state (block 1205). Forinstance, the user device 105 may obtain the user ID and extract theuser profile from the user data using the user ID; obtain the user stateas discussed above; and obtain the reaction tag from thetransition-coded media.

The user device 105 may then proceed to extract media paths fromreaction tag (block 1210). Reaction tags may include one or more mediapaths that provide one or more changes to content of the media element.Each media path may correspond to certain preferences for types ofmedia, sections of the media element, and/or objects of content of themedia element. For instance, the user device 105 may parse the reactiontag, and extract the media paths.

The user device 105 may then proceed to determine a media path based onthe user profile and the user state (block 1215). For instance, the userdevice 105 may determine whether any preferences from the userpreference data of the user data match any preferences of the mediapaths; if one media path matches a user preference, the user device 105may select that media path; if more than one media path matches, theuser device 105 may select one of the media paths based on priorityrules. The priority rules may select a media path corresponding to astrongest preference of a user's preference (by default) but, generally,are definable by the author.

The user device 105 may then proceed to update the media element withthe media path (block 1220). For instance, the user device 105 may alterthe media element to include the media path in place of the defaultmedia path.

Turning to flowchart 1300 for the author feedback process, the userdevice 105 may start the process of the flowchart 1300 by obtaining thereaction tag, a user profile, and a user state (block 1305). Forinstance, the user device 105 may obtain the user ID and extract theuser profile from the user data using the user ID; obtain the user stateas discussed above; and obtain the reaction tag from thetransition-coded media.

The user device 105 may then proceed to generate a reaction message(block 1310). For instance, the user device 105 may generate thereaction message based on the progress of the media element, the userstate, and/or the user profile. The reaction message may also includethe user ID and a component ID of the reaction tag.

The user device 105 may then proceed to transmit the reaction message toa service (block 1315). Once the reaction message is transmitted, theserver 115 may proceed as discussed in FIG. 18 below.

Examples of the systems and methods of this aspect of the disclosure:

Example 1: A user is reading a book (a transition-coded media) on theuser device 105. The book may have character tags associated withreaction tags for the character tags. The character tags may correspondto characters A-E at progress points indicating scenes in which one ormore characters A-E are present. As the user device 105 may have abaseline engagement established for the user, the reaction tags maymeasure deviations from the baseline for each scene with a charactertag. The deviations from the baseline may be a measure of an engagementintensity of the user for a character. For instance, the user device 105may obtain the user's state and compare values to the baseline to detectdeviations. To detect deviations, the user device 105 may determine anychange more than a deviation threshold (e.g., 5%) from the baseline is adeviation indicating an increase in engagement or a decrease inengagement (referred to as engagement intensity (EI) below).

The user device 105 may also generate EI profiles for characters/objectsbased on user inputs (e.g., in response to a prompt tag) and/or machinelearning models. The EI profiles may classify a user's reaction to acharacter/object as a particular human emotion or level interest. EIprofiles may inform the author how to best create atmospheric moods inmedia that drive different types of emotion.

TABLE 1 Summary of Example User EI and EI Profile Character/Object EI EIProfile Scene 1 Scene 2 . . . Scene N A (character tag A) +30% EI Crush60% −40%  . . . 60% B (character tag B) −50% EI Dislike −100%  10% . . .−100%  C (character tag C) +20% EI Admire 20% 50% . . . 20% B + C(character tag B + C) −60% EI Bored −200%  −80%  . . . −200%  D(character tag D) +20% EI Interested 10% 50% . . . 10% E (character tagE) +10% EI Intrigued 10% −5% . . . 10%

Example 2: On a display adjacent to an ATM information/images may bedisplayed (e.g., a card offering, highlighted by a celebrity orspokesperson) as transition-coded media. The ATM may have a camera, andthe ATM could be integrated into the display. During a plurality ofuser's interactions with the ATM, the camera (acting a tracking device)may detect that the users are looking at a particular section of theinformation/images on the display; a reaction tag may be triggered bythe users looking at the particular section of the information/images onthe display causing an author feedback process regarding the users'heightened interested in the particular section of theinformation/images on the display. The author of the information/imagesmay then use the information of the users' heightened interested tochange the information/images or more efficiently prepare futureinformation/images.

Example 3: Same as Example 2 above, but instead the information/imagesmay be different in different locations (or changed during periods ofthe day, etc.) and the transition-coded media may monitor user reactionsto the different information/images and/or a user's interest in aparticular section of the information/images. The author of theinformation/images may then use the user reactions to determine featuresof the different information/images to change the information/images touse a higher interest information/images or more efficiently preparefuture information/images.

Therefore, in this aspect of the disclosure, the systems and methods ofthe present disclosure may track a user's state and (1) perform actionson a conditional basis of the user's state; (2) change the content ofthe media file based on the user's state; and/or (3) provide reactiondata to the server 115. Therefore, the user device 105 may enableuser-specific experiences of transition-coded media and enable authorsto track how effective the media file and tags are causing users toreact. Additionally, the user device 105 may track the user's state todetermine how engaged the user is with the content (in general andcharacters/object/sections of media files in particular). Therefore,authors may be informed of how effective content of the media fileand/or a component (e.g., a particular tag) is at causing change in auser's state.

FIGS. 14-18 depict flowcharts for distribution of components oftransition-coded media, according to one or more embodiments. FIG. 14depicts a flowchart 1400 for distribution of components oftransition-coded media. Generally, the user device 105 (when acting asan authoring device) and the server 115 may cooperate to enabledistribution of components of transition-coded media.

The server 115 may start the process of the flowchart 1400 by hosting aservice for components of one or more transition-coded media (block1405). For instance, the server 115 may host the service as amarketplace to (1) gather and publish data of components oftransition-coded media and (2) distribute components for authors to usein generating transition-coded media.

The server 115 may then proceed to determine whether the service hasreceived a request from a user device (block 1410). For instance, theserver 115 may initiate processes in response to receiving messages fromuser devices, as discussed in more detail below. Generally, requests maybe messages or API calls, in accordance with an application programinterface format capable of being processed by the service.

In response to determining the service has not received a request from auser device (block 1410: No), the server 115 may return to determinewhether the service has received a request from a user device (block1410). In response to determining the service has received a requestfrom a user device (block 1410: Yes), the server 115 may then proceed todetermine whether the request includes a search request (block 1415).For instance, the server 115 may analyze the request and determinewhether the request includes a search indicator and, if so, determinethe request includes a search request. A search request may include oneor more search parameters with one or more logical operators linking theone or more search parameters.

In response to determining the request includes the search request(block 1415: Yes), the server 115 may then proceed to perform a searchprocess based on the search request (block 1420). For instance, theserver 115 may perform a search process as discussed below withreference to FIG. 15. After performing the search process, the server115 may proceed to return to determine whether the service has receiveda request from a user device (block 1410). In response to determiningthe request does not include the search request (block 1415: No), theserver 115 may proceed to determine whether the request includes acomponent request (block 1425). For instance, the server 115 may analyzethe request and determine whether the request includes a component serveindicator and, if so, determine the request includes a componentrequest. A component request may include one or more component IDs.

In response to determining the request includes the component request(block 1425: Yes), the server 115 may then proceed to perform acomponent serve process based on the component request (block 1430). Forinstance, the server 115 may perform a component serve process asdiscussed below with reference to FIG. 16. After performing thecomponent serve process, the server 115 may proceed to return todetermine whether the service has received a request from a user device(block 1410). In response to determining the request does not includethe component request (block 1415: No), the server 115 may proceed todetermine whether the request includes a data request (block 1435). Forinstance, the server 115 may analyze the request and determine whetherthe request includes a data serve indicator and, if so, determine therequest includes a data request. A data request may include one or morecomponent IDs.

In response to determining the request includes the data request (block1435: Yes), the server 115 may then proceed to perform a data serveprocess based on the data request (block 1440). For instance, the server115 may perform a data serve process as discussed below with referenceto FIG. 17. After performing the data serve process, the server 115 mayproceed to return to determine whether the service has received arequest from a user device (block 1410). In response to determining therequest does not include the data request (block 1415: No), the server115 may proceed to generate and transmit an error message (block 1445).After transmitting the error message, the server 115 may proceed toreturn to determine whether the service has received a request from auser device (block 1410).

FIG. 15 depicts a flowchart 1500 for a search process. The server 115may start the process of the flowchart 1500 by obtaining a searchrequest (block 1505). For instance, the server 115 may obtain the searchrequest by extracting the search request from the request received inFIG. 14 above.

The server 115 may then proceed to extract a search parameter from thesearch request (block 1510). For instance, the server 115 may parse thesearch request and obtain one or more search parameters. The searchparameters may include terms, IDs, words, and/or numbers (generally, analphanumeric string), for a search of a database of component IDs,descriptions, functionality, names, labels, etc. associated withcomponents of one or more transition-coded media.

The server 115 may then proceed to determine whether one or morecomponents have a matching parameter to the search parameter (block1515). For instance, the server 115 may access the database anddetermine whether an alphanumeric string of the component IDs,descriptions, functionality, names, labels match the alphanumeric stringof the search parameter. One of skill in the art would understand thatthe above description for a single search parameter is exemplary and notintended to exclude multiple search parameters linked by logicaloperators, or search in a free form manner using search parameters.

In response to determining none of the one or more components have amatching parameter to the search parameter (block 1515: No), the server115 may proceed to transmit a no match message to the user device 105(block 1520). For instance, the no match message may indicate that nocomponents have a matching parameter. In response to determining atleast one of the one or more components have a matching parameter to thesearch parameter (block 1515: Yes), the server 115 may then proceed togenerate a response based on the one or more components that include thematching parameter (block 1525). For instance, the server 115 maygenerate a search result page with information corresponding to the oneor more components that include the matching parameter. The server 115may then proceed to transmit the response to the user device 105 (block1530).

FIG. 16 depicts a flowchart 1600 for a component server process. Theserver 115 may start the process of the flowchart 1600 by obtaining acomponent request (block 1605). For instance, the server 115 may obtainthe component request by extracting the component request from therequest received in FIG. 14 above.

The server 115 may then proceed to extract a component ID from thecomponent request (block 1610). For instance, the server 115 may parsethe component request and obtain one or more component IDs. Thecomponent IDs may be terms, IDs, words, and/or numbers (generally, analphanumeric string), for a search of the database for component IDscorresponding to components of one or more transition-coded media.

The server 115 may then proceed to determine whether one or morecomponents have a matching ID to the component ID (block 1615). Forinstance, the server 115 may access the database and determine whetheran alphanumeric string of the component IDs of the database match thealphanumeric string of the component ID of the request. One of skill inthe art would understand that the above description for a singlecomponent ID is exemplary and not intended to exclude multiple componentIDs.

In response to determining none of the one or more components have amatching ID to the component ID (block 1615: No), the server 115 mayproceed to transmit a no match message to the user device 105 (block1620). For instance, the no match message may indicate that nocomponents have a matching component ID. In response to determining atleast one of the one or more components have a matching ID to thecomponent ID (block 1615: Yes), the server 115 may then proceed togenerate a response based on the one or more components that include thematching ID (block 1625). For instance, the server 115 may generate asearch result page with information corresponding to the one or morecomponents that include the matching ID, and/or transmit theframework(s) for the one or more components that include the matchingID. The server 115 may then proceed to transmit the response to the userdevice 105 (block 1630).

FIG. 17 depicts a flowchart 1700 for a data serve process. The server115 may start the process of the flowchart 1700 by obtaining a datarequest (block 1705). For instance, the server 115 may obtain the datarequest by extracting the data request from the request received in FIG.14 above.

The server 115 may then proceed to extract a component ID from thecomponent request (block 1710). For instance, the server 115 may parsethe component request and obtain one or more component IDs. Thecomponent IDs may be terms, IDs, words, and/or numbers (generally, analphanumeric string), for a search of the database for component IDscorresponding to components of one or more transition-coded media.

The server 115 may then proceed to determine whether one or morecomponents have a matching ID to the component ID (block 1715). Forinstance, the server 115 may access the database and determine whetheran alphanumeric string of the component IDs of the database match thealphanumeric string of the component ID of the request. One of skill inthe art would understand that the above description for a singlecomponent ID is exemplary and not intended to exclude multiple componentIDs.

In response to determining none of the one or more components have amatching ID to the component ID (block 1715: No), the server 115 mayproceed to transmit a no match message to the user device 105 (block1720). For instance, the no match message may indicate that nocomponents have a matching component ID. In response to determining atleast one of the one or more components have a matching ID to thecomponent ID (block 1715: Yes), the server 115 may then obtain theanalytics data for the one or more components that include the matchingID (block 1725). The analytics data may include the media engagementdata for users who have interacted with a transition-coded media.

The server 115 may then proceed to generate a response based on theanalytics data (block 1730). For instance, the server 115 may generate adata display page with information based on the analytics datacorresponding to the one or more components that include the matchingID, and/or a report of the analytics data for the one or more componentsthat include the matching ID to be consumed by a third-party process(e.g., billing, membership, etc.). The server 115 may then proceed totransmit the response to the user device 105 (block 1735).

Therefore, in this aspect of the disclosure, the systems and methods ofthe present disclosure may enable authors to search for and requestcomponents, and/or search for and request data regarding users/userreactions to components. The distributed components may enable authorsto include tags into their transition-coded media to enhance a user'sexperience. Therefore, transition-coded media may be developed quicklyand tracked for effectiveness.

FIG. 18 depicts a flowchart 1800 for updating analytics data. The server115 may start the process of the flowchart 1800 by determining whetherthe service has received a data input message from a user device (block1805). In response to determining the service has not received a datainput message from a user device (block 1805: No), the server 115 mayproceed to return to determine whether the service has received a datainput message from a user device (block 1805). For instance, the server115 may analyze an incoming request and determine whether the requestincludes a data input serve indicator and, if so, determine the datainput message has been received. The data input message may include auser ID, one or more component IDs corresponding to components oftransition-coded media (e.g., a reaction tag of a media element), andnew media engagement data. Generally, the data input message may betransmitted by the user device 105 to effect the author feedback type ofreaction tags. The data input message may be each reaction messagetransmitted as the reaction tags are triggered, or the author feedbacktype may be a batch message including multiple reaction messages formultiple reaction tags transmitted every set period of time (e.g., oncea day). In the case of the batch message, each of the multiple reactionmessages may be processed separately (in sequence or in parallel) orjointly.

In response to determining the service has received a data input messagefrom a user device (block 1805: Yes), the server 115 may then proceed toextract a component ID from the data input message (block 1810). Forinstance, the server 115 may parse the data input message and obtain oneor more component IDs. The component IDs may be terms, IDs, words,and/or numbers (generally, an alphanumeric string), for a search of thedatabase for component IDs corresponding to components of one or moretransition-coded media, so that the analytics data of the components maybe updated.

The server 115 may then proceed to determine whether one or morecomponents have a matching ID to the component ID (block 1815). Forinstance, the server 115 may access the database and determine whetheran alphanumeric string of the component IDs of the database match thealphanumeric string of the component ID of the request. One of skill inthe art would understand that the above description for a singlecomponent ID is exemplary and not intended to exclude multiple componentIDs.

In response to determining none of the one or more components have amatching ID to the component ID (block 1815: No), the server 115 mayproceed to transmit a no match message to the user device 105 (block1820). For instance, the no match message may indicate that nocomponents have a matching component ID. In response to determining atleast one of the one or more components have a matching ID to thecomponent ID (block 1815: Yes), the server 115 may then proceed toextract the new media engagement data from the data input message (block1825). For instance, the server 115 may parse the data input message andobtain the new media engagement data.

The server 115 may then proceed to obtain media engagement data for theone or more components that include the matching ID (block 1830). Forinstance, the server 115 may obtain the analytics data for the one ormore components that include the matching ID, and extract the mediaengagement data.

The server 115 may then proceed to update the media engagement data forthe one or more components that include the matching ID based on the newmedia engagement data extracted from the data input message (block1835). For instance, the server 115 may update/add a dataset to the oneor more reaction datasets. Specifically, the server 115 may extract auser ID from the data input message; determine whether a dataset of theplurality of datasets is associated with the user ID and the componentID by matching the user ID of the data input message to user IDs of thedatasets and matching component ID of the data input message tocomponent IDs for a component ID. If both the user ID and component IDmatch a dataset, the server 115 may update the matching dataset. If onlyone or neither of user ID and component ID match a dataset, the server115 may add a dataset. For instance, to update the dataset, the server115 may save each user reaction to a reaction tag, save a particularnumber of most recent reactions, or save only a most recent reaction toa tag. The server 115 may also update the averages and/or scores for thecomponent ID based on the updated dataset or added dataset.

The server 115 may then proceed to generate report(s) based on updatedmedia engagement data (block 1840). For instance, the server 115 maygenerate the reports at set intervals time, at set numbers ofnew/updated datasets for a component ID (e.g., after 100 new/updatedreaction datasets by tracking a number of times the datasets for thecomponent ID are updated/added), and/or in response to a new/updateddataset. The reports may include the averages/scores of the reactiondata for the component ID, and include user summary information. Theuser summary information may include a number of distinct users thatthat have engaged with the component, a number of reactions to thecomponent, and/or user characteristics of users that that have engagedwith the component. The user characteristics may include age, gender,location, profession, etc.

The server 115 may then proceed to transmit the report message(s) toowner(s) (block 1845). The report messages may include the reports. Forinstance, the component ID may be associated with one or more user IDsthat indicates an owner of the component on the service hosted by theserver 115. An owner may be an original author of a component and/or anauthor of a transition-coded media element that includes the component.Additionally, the server 115 may update published data for thecomponents corresponding to the component ID on the service.

Therefore, in this aspect of the disclosure, the systems and methods ofthe present disclosure may gather and publish data for components oftransition-coded media, and/or gather and inform authors of user'sreactions to transition-coded media. Authors may use the information todetermine a success invoking particular user reactions and/or beinformed of how engaged users are with content of the media file.

FIG. 19 depicts an example system that may execute techniques presentedherein. FIG. 19 is a simplified functional block diagram of a computerthat may be configured to execute techniques described herein, accordingto exemplary embodiments of the present disclosure. Specifically, thecomputer (or “platform” as it may not be a single physical computerinfrastructure) may include a data communication interface 1960 forpacket data communication. The platform may also include a centralprocessing unit (“CPU”) 1920, in the form of one or more processors, forexecuting program instructions. The platform may include an internalcommunication bus 1910, and the platform may also include a programstorage and/or a data storage for various data files to be processedand/or communicated by the platform such as ROM 1930 and RAM 1940,although the system 1900 may receive programming and data via networkcommunications. The system 1900 also may include input and output ports1950 to connect with input and output devices such as keyboards, mice,touchscreens, monitors, displays, etc. Of course, the various systemfunctions may be implemented in a distributed fashion on a number ofsimilar platforms, to distribute the processing load. Alternatively, thesystems may be implemented by appropriate programming of one computerhardware platform.

The general discussion of this disclosure provides a brief, generaldescription of a suitable computing environment in which the presentdisclosure may be implemented. In one embodiment, any of the disclosedsystems, methods, and/or graphical user interfaces may be executed by orimplemented by a computing system consistent with or similar to thatdepicted and/or explained in this disclosure. Although not required,aspects of the present disclosure are described in the context ofcomputer-executable instructions, such as routines executed by a dataprocessing device, e.g., a server computer, wireless device, and/orpersonal computer. Those skilled in the relevant art will appreciatethat aspects of the present disclosure can be practiced with othercommunications, data processing, or computer system configurations,including: Internet appliances, hand-held devices (including personaldigital assistants (“PDAs”)), wearable computers, all manner of cellularor mobile phones (including Voice over IP (“VoIP”) phones), dumbterminals, media players, gaming devices, virtual reality devices,multi-processor systems, microprocessor-based or programmable consumerelectronics, set-top boxes, network PCs, mini-computers, mainframecomputers, and the like. Indeed, the terms “computer,” “server,” and thelike, are generally used interchangeably herein, and refer to any of theabove devices and systems, as well as any data processor.

Aspects of the present disclosure may be embodied in a special purposecomputer and/or data processor that is specifically programmed,configured, and/or constructed to perform one or more of thecomputer-executable instructions explained in detail herein. Whileaspects of the present disclosure, such as certain functions, aredescribed as being performed exclusively on a single device, the presentdisclosure may also be practiced in distributed environments wherefunctions or modules are shared among disparate processing devices,which are linked through a communications network, such as a Local AreaNetwork (“LAN”), Wide Area Network (“WAN”), and/or the Internet.Similarly, techniques presented herein as involving multiple devices maybe implemented in a single device. In a distributed computingenvironment, program modules may be located in both local and/or remotememory storage devices.

Aspects of the present disclosure may be stored and/or distributed onnon-transitory computer-readable media, including magnetically oroptically readable computer discs, hard-wired or preprogrammed chips(e.g., EEPROM semiconductor chips), nanotechnology memory, biologicalmemory, or other data storage media. Alternatively, computer implementedinstructions, data structures, screen displays, and other data underaspects of the present disclosure may be distributed over the Internetand/or over other networks (including wireless networks), on apropagated signal on a propagation medium (e.g., an electromagneticwave(s), a sound wave, etc.) over a period of time, and/or they may beprovided on any analog or digital network (packet switched, circuitswitched, or other scheme).

Program aspects of the technology may be thought of as “products” or“articles of manufacture” typically in the form of executable codeand/or associated data that is carried on or embodied in a type ofmachine-readable medium. “Storage” type media include any or all of thetangible memory of the computers, processors or the like, or associatedmodules thereof, such as various semiconductor memories, tape drives,disk drives and the like, which may provide non-transitory storage atany time for the software programming. All or portions of the softwaremay at times be communicated through the Internet or various othertelecommunication networks. Such communications, for example, may enableloading of the software from one computer or processor into another, forexample, from a management server or host computer of the mobilecommunication network into the computer platform of a server and/or froma server to the mobile device. Thus, another type of media that may bearthe software elements includes optical, electrical and electromagneticwaves, such as used across physical interfaces between local devices,through wired and optical landline networks and over various air-links.The physical elements that carry such waves, such as wired or wirelesslinks, optical links, or the like, also may be considered as mediabearing the software. As used herein, unless restricted tonon-transitory, tangible “storage” media, terms such as computer ormachine “readable medium” refer to any medium that participates inproviding instructions to a processor for execution.

Other embodiments of the disclosure will be apparent to those skilled inthe art from consideration of the specification and practice of theinvention disclosed herein. It is intended that the specification andexamples be considered as exemplary only, with a true scope and spiritof the invention being indicated by the following claims.

1. A method for transition-coded media, comprising: obtaining a mediafile; determining whether the media file includes transition-codedmedia, the transition-coded media including a media element and one ormore tags, the one or more tags being associated with one or moreprogress points in the media element; in response to determining themedia file includes the transition-coded media, establishing a dynamicmedia environment; detecting whether a user is interacting with thetransition-coded media; and in response to detecting the user isinteracting with the transition-coded media, executing thetransition-coded media with the dynamic media environment by: tracking aprogress of the media element and a user state; based on the progress ofthe media element and the user state, determining whether a triggerevent occurs, the trigger event being associated with a tag of the oneor more tags; in response to determining the trigger event occurs,determining a transition action based on the trigger event by:extracting information for one or more device actions and informationfor one or more tag devices from the tag corresponding to the triggerevent; comparing the information for the one or more tag devices toinformation for available dynamic media environment enabled devices;determining matching devices based on a comparison result of thecomparing the information for the one or more tag devices to theinformation for the available dynamic media environment enabled devices;and determining the transition action based on the matching devices andthe one or more device actions; and performing the transition action. 2.(canceled)
 3. The method of claim 1, wherein the tracking the progressof the media element and the user state includes: sending instructionsto at least one tracking device to track the user; receiving trackingdata from the at least one tracking device; and processing the trackingdata to determine the user state.
 4. The method of claim 3, wherein thetracking the progress of the media element and the user state furtherincludes: determining whether the media element is a static mediaelement or data stream media element, and performing one of: in responseto determining the media element is a static media element, determiningthe progress of the media element based on the user state; or inresponse to determining the media element is a data stream mediaelement, determining the progress of the media element based on time. 5.The method of claim 1, wherein the determining whether the trigger eventoccurs includes: determining whether a progress point of the tag hasbeen reached based on the progress of the media element; and in responseto determining the progress point of the tag has been reached,determining the trigger event occurs.
 6. The method of claim 5, whereinthe determining the transition action based on the trigger eventincludes: determining whether the tag includes a condition; and inresponse to determining the tag does not include a condition,determining the transition action based on the tag.
 7. The method ofclaim 5, wherein the determining the transition action based on thetrigger event includes: determining whether the tag includes acondition; in response to determining the tag does include a condition,determining whether the condition is satisfied based on the progress ofthe media element and/or the user state; and in response to determiningthe condition is satisfied, determining the transition action based onthe tag.
 8. (canceled)
 9. The method of claim 1, wherein the performingthe transition action includes: sending instructions to the matchingdevices to perform at least one device action of the one or more deviceactions in accordance with the tag.
 10. The method of claim 1, whereinthe establishing the dynamic media environment includes: obtaininglocation data for the dynamic media environment, the location dataincluding information about one or more dynamic media environmentenabled devices; and initializing the one or more dynamic mediaenvironment enabled devices.
 11. The method of claim 10, theestablishing the dynamic media environment further includes, before theobtaining the location data: determining a location of the dynamic mediaenvironment; determining whether the location is one of one or moreknown locations; in response to determining the location is one of oneor more known locations, obtaining the location data stored in a memory;and in response to determining the location is not one of one or moreknown locations, executing a search process for dynamic mediaenvironment enabled devices to generate the location data.
 12. A systemfor transition-coded media, the system comprising: a memory storinginstructions; and a processor executing the instructions to perform aprocess including: obtaining a media file; determining whether the mediafile includes transition-coded media, the transition-coded mediaincluding a media element and one or more tags, the one or more tagsbeing associated with one or more progress points in the media element;in response to determining the media file includes the transition-codedmedia, establishing a dynamic media environment by: obtaining locationdata for the dynamic media environment, the location data includinginformation about one or more dynamic media environment enabled devices,and initializing the one or more dynamic media environment enableddevices; detecting whether a user is interacting with thetransition-coded media; and in response to detecting the user isinteracting with the transition-coded media, executing thetransition-coded media with the dynamic media environment by: tracking aprogress of the media element and a user state; based on the progress ofthe media element and the user state, determining whether a triggerevent occurs, the trigger event being associated with a tag of the oneor more tags; in response to determining the trigger event occurs,determining a transition action based on the trigger event by:extracting information for one or more device actions and informationfor one or more tag devices from the tag corresponding to the triggerevent; comparing the information for the one or more tag devices toinformation for available dynamic media environment enabled devices;determining matching devices based on a comparison result of thecomparing the information for the one or more tag devices to theinformation for the available dynamic media environment enabled devices;and determining the transition action based on the matching devices andthe one or more device actions; and performing the transition action.13. (canceled)
 14. The system of claim 12, wherein the tracking theprogress of the media element and the user state includes: sendinginstructions to at least one tracking device to track the user;receiving tracking data from the at least one tracking device; andprocessing the tracking data to determine the user state.
 15. The systemof claim 14, wherein the tracking the progress of the media element andthe user state further includes: determining whether the media elementis a static media element or data stream media element, and performingone of: in response to determining the media element is a static mediaelement, determining the progress of the media element based on the userstate; or in response to determining the media element is a data streammedia element, determining the progress of the media element based ontime.
 16. The system of claim 12, wherein the determining whether thetrigger event occurs includes: determining whether a progress point ofthe tag has been reached based on the progress of the media element; andin response to determining the progress point of the tag has beenreached, determining the trigger event occurs.
 17. The system of claim16, wherein the determining the transition action based on the triggerevent includes: determining whether the tag includes a condition; inresponse to determining the tag does include a condition, determiningwhether the condition is satisfied based on the progress of the mediaelement and/or the user state; and in response to determining thecondition is satisfied, determining the transition action based on thetag.
 18. (canceled)
 19. The system of claim 12, wherein the performingthe transition action includes: sending instructions to the matchingdevices to perform at least one device action of the one or more deviceactions in accordance with the tag.
 20. A method for transition-codedmedia, the method comprising: obtaining a media file; determiningwhether the media file includes transition-coded media, thetransition-coded media including a media element and one or more tags,the one or more tags being associated with one or more progress pointsin the media element; in response to determining the media file includesthe transition-coded media, establishing a dynamic media environment;detecting whether a user is interacting with the transition-coded media;and in response to detecting the user is interacting with thetransition-coded media, executing the transition-coded media with thedynamic media environment by: tracking a progress of a media element ofthe transition-coded media and a user state; based on the progress ofthe media element and the user state, determining whether a triggerevent occurs; in response to determining the trigger event occurs,determining a transition action based on the trigger event by extractinginformation for one or more device actions and information for one ormore tag devices from a tag corresponding to the trigger event;comparing the information for the one or more tag devices to informationfor available dynamic media environment enabled devices; determiningmatching devices based on a comparison result of the comparing theinformation for the one or more tag devices to the information for theavailable dynamic media environment enabled devices; and determining thetransition action based on the matching devices and the one or moredevice actions; and performing the transition action.